Video generation device, video generation method, and recording medium

ABSTRACT

A video is superimposed on an object in order that the object will be perceived as if the object were given a motion. This video is a video including a luminance motion component corresponding to a motion given to the object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a division of and claims the benefit of priorityunder 35 U.S.C. § 120 from U.S. application Ser. No. 16/739,414 filedJan. 10, 2020, which is a division of U.S. application Ser. No.15/306,011 filed Oct. 21, 2016 (now U.S. Pat. No. 10,571,794 issued Feb.25, 2020), the entire contents of which are incorporated herein byreference. U.S. application Ser. No. 15/306,011 is a National Stage ofPCT/JP15/062093 filed Apr. 21, 2015, which claims the benefit forpriority under 35 U.S.C. § 119 from Japanese Application No. 2014-088389filed Apr. 22, 2014 and Japanese Application No. 2014-230720 filed Nov.13, 2014.

TECHNICAL FIELD

The present invention relates to a technique for providing visualillusion.

BACKGROUND ART

Non-patent Literature 1 discloses a technique in which athree-dimensional shape of an object (single achromatic color) ismeasured with a camera and a video representing a motion is projected inconformity with the three-dimensional shape of the object so as to givean effect of an illusory motion to the object. In Non-patent Literature1, the effect of a motion is given to an object such that a flatsingle-achromatic plane and a single-achromatic automobile model whichis placed on the plane are set as an object which serves as a canvas anda color scheme which is obtained by simulating colors and reflectionproperties of a body of an automobile, roads around the automobile,conditions of daylight in a space in which the automobile travels, andthe like is projected as a video onto the object so as to provide anillusion as if the automobile model which is the object travels roads.

PRIOR ART LITERATURE Non-Patent Literature

-   -   Non-patent Literature 1: Raskar, R.; Ziegler, R.; Willwacher,        T., “Cartoon Dioramas in Motion”, International Symposium on        Non-Photorealistic Animation and Rendering (NPAR), June 2002.

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In Non-patent Literature 1, the object is assumed merely as a canvas andthe effect of a motion is provided to the object without using a patternof the object. An object of the present invention is to allow an objectto be perceived as if the object were given a motion, by using a patternof the object.

Means to Solve the Problems

In order that an object will be perceived as if the object were given amotion, a video is superimposed on the object. This video includes aluminance motion component corresponding to the motion given to theobject.

Effects of the Invention

Accordingly, the object can be perceived as if the object were given amotion by using a pattern of the object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of a videodisplay device according to a first embodiment.

FIG. 2 is a flowchart illustrating an operation of the video displaydevice according to the first embodiment.

FIG. 3 is a block diagram illustrating the configuration of a videoprojection device according to an example of the first embodiment.

FIG. 4 is a flowchart illustrating an operation of the video projectiondevice according to the example of the first embodiment.

FIG. 5 is a diagram for explaming a relationship between a projectionimage angle and an object in the case where a projection unit of thevideo projection device according to the example of the first embodimentis realized by a projector.

FIG. 6 is a block diagram illustrating the configuration of a videodisplay device according to a second embodiment.

FIG. 7 is a flowchart illustrating an operation of the video displaydevice according to the second embodiment.

FIG. 8 is a block diagram illustrating the configuration of a videodisplay device according to a third embodiment.

FIG. 9 is a flowchart illustrating an operation of a video generationunit of the video display device according to the third embodiment.

FIG. 10 is a diagram illustrating a static component and a motioncomponent.

FIG. 11 is a block diagram illustrating the configuration of a videodisplay device according to a fourth embodiment.

FIG. 12 is a flowchart illustrating an operation of the video displaydevice according to the fourth embodiment.

FIG. 13A and FIG. 13B are conceptual diagrams for illustrating anoutline of processing.

FIG. 14A and FIG. 14B are conceptual diagrams for illustrating anoutline of processing.

FIG. 15 is a block diagram illustrating the functional configuration ofthe embodiments.

FIG. 16 is a flow diagram illustrating processing of the embodiments.

FIG. 17 is a flow diagram illustrating processing of the embodiments.

FIG. 18A and FIG. 18B are diagrams for illustrating a method for puttinga luminance motion component on a picture image.

FIG. 19 is a block diagram illustrating the functional configuration ofthe embodiments.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described. In theembodiments described below, an appearance of a pattern is deformedillusorily while using the pattern of an object unlike the previoustechniques. In each of the embodiments, an object is not merely assumedas a canvas, but a pattern of the object is actively used to create anillusion of a motion. Accordingly, the object is not set to besingle-achromatic but preferably set to be chromatic or in thegrayscale. In each of the embodiments, three-dimensionality of an objectdoes not have to be considered unlike the related art. For example, anappearance of a picture image obtained by photographing a static objectcan also be deformed illusorily. In the related art, aspects in transferand deformation of an object have been focused. However, both of apattern of an object and image information of a video which issuperimposed and displayed on the object are used to provide the objectwith textures and impressions which do not originally exist on theobject (an impression of liquid or facial expressions) in each of theembodiments. Thus, the embodiments are different from the previoustechniques on this point. Further, different from the technique formerely deforming a taken picture image on a screen, texture impressionchange of an object can be provided to viewers by superposition of avideo. Accordingly, perceptual experiences different from those of therelated art can be provided to viewers. Each of the embodiments will bedescribed in detail below. Here, constituent elements having the samefunctions will be given the same reference numerals and duplicateddescription will be omitted.

First Embodiment

A video display device, according to a first embodiment, which is abasic configuration example of the present invention is described belowwith reference to FIGS. 1 and 2. FIG. 1 is a block diagram illustratingthe configuration of a video display device 1 according to the presentembodiment. FIG. 2 is a flowchart illustrating an operation of the videodisplay device 1 according to the present embodiment. As illustrated inFIG. 1, the video display device 1 includes a display unit 11. Thedisplay unit 11 superimposes and displays a video having transparency onan object. Here, the “object” may be a thing having a three-dimensionalshape (a vase, a ball, or a model, for example) or a predetermined plane(a paper, a board, a wall, or a screen, for example). In the case wherethe object is a plane, a pattern is preferably included in the plane.Possible examples of the pattern included in the plane include aphotograph and a picture image printed on a paper and a photograph and apicture image projected onto a predetermined plane. In the case where ascreen such as a display is set as the object, possible examples of thepattern include a picture image displayed on the screen such as adisplay.

Examples of the method for “superposing and displaying” a video havingtransparency on an object include a method for projecting a video ontothe object by a projector or the like as a typical method. In this case,the video projected by a projector or the like naturally hastransparency. Further, a transparent liquid crystal screen may be placedin front of an object and a video having transparency may be distributedon the liquid crystal screen so as to “superimpose and display” a videohaving transparency on the object when viewed from a viewer opposed tothe object with the liquid crystal screen interposed, for example. Here,the “video” represents a video in which picture images, which havedistortion distribution including low spatial frequency components, aretemporally switched, for example. In the present embodiment, it isassumed that a video is prepared in advance and is input from theoutside of the video display device 1. The display unit 11 displays avideo so that an edge included in the video is overlapped with anoutline of the object or on an edge included in the object. For example,in the case where the object is a vase, the display unit 11 displays avideo so that an edge included in the video is overlapped with anoutline of the vase which is the object or an edge included in the vasesuch as a pattern drawn on the vase. For example, in the case where theobject is a plane and a picture image is projected as a pattern on theplane, the display unit 11 displays a video so that an edge included inthe video is overlapped with an edge included in the object such as apattern of the picture image projected on the object.

Thus, the display unit 11 superimposes and displays a video, which hastransparency and in which picture images having distortion distributionincluding low spatial frequency components are temporally switched, onthe object so that an edge included in the video is overlapped with anoutline of the object or an edge included in the object (S11).

Example of First Embodiment

A video projection device 1 a which is an example of the firstembodiment will be described below with reference to FIGS. 3 and 4. FIG.3 is a block diagram illustrating the configuration of the videoprojection device 1 a which is the example of the present embodiment.FIG. 4 is a flowchart illustrating an operation of the video projectiondevice 1 a which is the example of the present embodiment. Asillustrated in FIG. 3, the video projection device 1 a is configured toinclude a projection unit 11 a. In the present example, a static objectwill be discussed. In a similar manner to the above description, theprojection unit 11 a superimposes and projects a video, in which pictureimages having distortion distribution including low spatial frequencycomponents are temporally switched, on the static object so that an edgeincluded in the video is overlapped with an outline of the object or anedge included in the object (S11 a). The projection unit 11 a can berealized by a projector, for example. In the case where the projectionunit 11 a is a projector, for example, a viewing angle θ in thehorizontal direction and a viewing angle 41 in the vertical direction,which are viewed from the center of a projection lens of the projector,of an object 9 are required to be accorded with viewing angles in thehorizontal and vertical directions of the projected video as illustratedin FIG. 5.

Here, the projection unit 11 a can be similarly adopted as an example ofthe display unit 11 also in second, third, and fourth embodiments, whichwill be described later, and each block diagram illustrating eachcorresponding device of the later-described embodiment illustrates thatthe projection unit 11 a is included as the example of the display unit11.

In the technique of Non-patent Literature 1, it is not easy to provide aspecial illusion showing as if a medium (for example, air, water, orwater vapor) around an object wavered irregularly. Provision of anillusion as the above-described special illusion by applying thetechnique of Non-patent Literature 1 requires enormous calculation forsimulating wavering of a medium, refraction of a ray of light, and thelike. According to the video projection device 1 a of the presentembodiment, a special illusion can be given to an object with a smallcalculation amount. Further, according to the video projection device 1a of the present embodiment, an illusion can be given to an objecthaving a planar shape, an object having a three-dimensional shape, achromatic object, and an achromatic object having variations in contrastdensity as well.

Second Embodiment

A video display device 2, according to the second embodiment, includinga video generation function therein will be described below withreference to FIGS. 6 and 7. FIG. 6 is a block diagram illustrating theconfiguration of the video display device 2 according to the presentembodiment. FIG. 7 is a flowchart illustrating an operation of the videodisplay device 2 according to the present embodiment. As illustrated inFIG. 6, the video display device 2 includes a display unit 11 which issame as that of the first embodiment, a photographing unit 21, and avideo generation unit 22.

The photographing unit 21 photographs an object so as to acquire anoriginal image (S21). The video generation unit 22 generates severalnarrow-band images, which are different from each other, from theoriginal image so as to generate a video in which the narrow-band imageswhich are generated and different from each other are arranged to betemporally smoothly continued (S22). The display unit 11 superimposesand displays the generated video on the object as is the case with theabove description (S11).

The narrow-band image is a picture image which is obtained such that aspatial frequency band of the whole picture image is narrowed to benarrower than the spatial frequency band of the original image whilemaintaming information of an edge included in the original image, andfurther, the narrow-band image is a picture image having transparency.

As an implementation of the video generation unit 22, picture images maybe convoluted by an orientation filter a phase of which is differentfrom those of the picture images by 180 degrees and the convolutedpicture images may be temporally smoothly continued so as to generate avideo which provides a motional impression illusorily, as thedescription of Reference Literature 1, for example.

(Reference Literature 1: Freeman, W. T., Adelson, E. H., & Heeger, D. J.(1991). Proceedings of the 18th annual conference on computer graphicsand interactive techniques, 27-30.)

Third Embodiment

A video display device 3, according to the third embodiment, including avideo generation function therein will be described below with referenceto FIGS. 8 and 9. FIG. 8 is a block diagram illustrating theconfiguration of the video display device 3 according to the presentembodiment. FIG. 9 is a flowchart illustrating an operation of a videogeneration unit 32 of the video display device 3 according to thepresent embodiment. As illustrated in FIG. 8, the video display device 3includes a display unit 11 and a photographing unit 21 which are same asthose of the second embodiment and the video generation unit 32 which isdifferent from that of the second embodiment. The video generation unit32 includes a deformed video generation unit 321, a Fourier transformunit 322, and a separation unit 323. The deformed video generation unit321 applies dynamic deformation to an original image obtained byphotographing an object so as to generate a video in which a staticobject image is deformed (S321). The deformation is expressed by pixelwarp. This pixel warp is based on an algorithm which is preliminarilycalculated. In the case where a liquid impression is desired to be givento an object, for example, algorithms of Reference Literature 2 andJapanese Patent Application No. 2013-132609 which is a patentapplication which had not been published at the time of filing of theconcerned application are used as a reference.

(Reference Literature 2: Kawabe, T., Maruya, K., & Nishida, S. (2013).Seeing transparent liquids from dynamic image distortion. Journal ofVision, 13(9): 208.)

An implementation of the deformed video generation unit 321 based on thealgorithm of Japanese Patent Application No. 2013-132609 is disclosedbelow. The deformed video generation unit 321 first generates severalmodulated images obtained by modulating an original image based ondistortion distribution. At this time, in order to provide strongperception of an impression for flow of transparent liquid, it ispreferable that the original image is modulated with distortion so thata spatial frequency of distortion distribution (distortion map) is equalto or smaller than 3 cpd (cycles per degree). In other words, when roughdistortion, by which a gap of distortion amounts between adjacent pixelsbecomes smaller, is applied to the original image, an impression oftransparent liquid can be further strongly perceived. Here, the originalimage may be modulated by using distortion distribution includingidentical low spatial frequency components (for example, 3 cpd orsmaller) or by using distortion distribution including different lowspatial frequency components (for example, 2 cpd or smaller for one and3 cpd or smaller for the other). Further, two or more dimensionaldistortion directions are given to the modulated images. Any distortionmay be employed as long as the distortion is two-dimensional geometricdistortion such as rotation distortion, parallel movement distortion,and random distortion.

Then, the deformed video generation unit 321 generates a video based onseveral modulated images which are generated from the original image.The deformed video generation unit 321 may generate a modulated imagesequence in which several modulated images generated from the originalimage are ordered so as to be temporally switched and presented as avideo, for example. The modulated image sequence is a sequence in whichpresentation time (frame rate) of each picture image is set in the rangethat viewers can view the modulated image sequence not as a sequence ofstatic images but as a moving image, that is, a “video which isstructured by temporally lming up modulated images”. Further, thedeformed video generation unit 321 may perform control for switching andpresenting each of several modulated images generated from the originalimage, for example. The presentation interval of each picture image maybe controlled in the range that viewers can view the modulated imagesnot as a sequence of static images but as a moving image (a video). Thepresentation time of each picture image may be set within 0.05 (sec) orthe frame rate may be set to be 20 Hz or larger, for example.

Then, the Fourier transform unit 322 applies three-dimensional(spatiotemporal) Fourier transform with respect to the video which isgenerated by the deformed video generation unit 321 (S322). Theseparation unit 323 separates DC components (static components) frommotion components by temporal filtering so as to output only the motioncomponents as a video (S323).

In another expression, the deformed video generation unit 321 generatesa video in which a group of several picture images obtained by warpingpixels of the original image based on a predetermined algorithm is linedup so that the picture images are continued temporally smoothly (S321).The Fourier transform unit 322 applies three-dimensional(spatiotemporal) Fourier transform with respect to the generated video(S322). The separation unit 323 separates static components from motioncomponents based on the result of the Fourier transform so as to outputonly the motion components as a video (S323).

FIG. 10 illustrates examples of the static component (DC component) andthe motion component which are extracted through the above-mentionedsteps S321 to S323. The motion components are mainly constituted of ahigh spatial frequency component. As illustrated in FIG. 5, the motioncomponents are superimposed and displayed, as a video, on an object in aviewing angle same as that of the object and thus, the phaserelationship of the two is accorded with the original moving image.Under this condition, the spatial structure of the object is illusorilycaptured due to the motion components. As a result, the object looksmoving.

Fourth Embodiment

A video display device 4, according to the fourth embodiment, includinga function therein for generating an original image which is to bedisplayed on an object will be described below with reference to FIGS.11 and 12. FIG. 11 is a block diagram illustrating the configuration ofthe video display device 4 according to the present embodiment. FIG. 12is a flowchart illustrating an operation of the video display device 4according to the present embodiment. As illustrated in FIG. 11, thevideo display device 4 of the present embodiment includes a display unit11, a photographing unit 21, and a video generation unit 32 which aresame as those of the third embodiment and an original image display unit41 which is not included in the third embodiment. The photographing unit21 and the video generation unit 32 respectively execute steps S21 andS32 as is the case with the third embodiment. The original image displayunit 41 displays (projects) an original image which is acquired byphotographing an object in step S21 on another static medium (forexample, a display, a screen, or the like) (S41). The display unit 11superimposes and displays (projects) a video with respect to theoriginal image, which is thus displayed on the static medium.Accordingly, the effect same as that in the above description isobtained. At this time, the video may be generated by convoluting anorientation filter with respect to picture images as described in thesecond embodiment or may be generated by deforming picture images asdescribed in the third embodiment. In the case of the display on anotherstatic medium, the original image can be printed on an object, projectedby another projector, or displayed on an electronic paper, for example.

An external camera is used for photographing of an object. The camerapreferably exhibits spatial resolution as high as possible. Except forcalculation time, an illusory action can be instantaneously added to astatic object located in front of eyes. As a projection unit 11 a of avideo projection device 4 a which is an example of the presentembodiment, a projector can be used. A commercially available projectormay be used as this projector. However, in the case where the projectoris used in a bright room, a projector emitting high luminance isrequired.

An ideal viewing distance varies depending on the size of an object. Inthe case where a motion is given to a video having 16 square centimetersby using the method of the present embodiment, for example, the viewingdistance of approximately 1.5 meters is required. As an object islarger, the viewing distance is required to be set longer.

A gap may be generated between structures of a video and an object inthe case where the video is projected onto a three-dimensional object,in the case where projection is performed without considering the depthof an object projection surface, and in the case where the depthdifference on the object projection surface is large. It is morepreferable to use an object whose depth difference on the projectionsurface is relatively small to enable projection within a range where agap is not generated. Further, in the case where a video is projectedonto a three-dimensional object having the large depth, a surface tiltedtoward the depth direction can be deformed illusorily by grasping thethree-dimensional shape of the object and deforming videos to beprojected in accordance with the shape.

According to the video display device of the present embodiment, anexpression of a static face image can be changed illusorily or thedirection of eyes can be changed illusorily, for example. Further, bychanging a pattern of a motion, illusions can be provided that an objectis flapping and that the object exists under flowing liquid. In the casewhere a video is projected onto an object which is a two-dimensionalmedium such as a printed material, a dynamic range of luminance of theobject in the medium is generally narrow. However, a video having thedynamic range higher than that of the object is projected onto theobject in a superimposed manner in the present embodiment, so that anillusory action can be added to the object while enhancing the imagequality of an appearance of the object. Further, according to the videodisplay device of the present embodiment, an illusory effect of a motioncan be instantaneously added to a picture image photographed by acamera, except for processing time.

The video display device of the present embodiment is applicable for anexhibition technique in an art museum or a museum or as an elementtechnique of attractions used in entertainment facilities. For example,characters for children printed on a static medium such as a paper canbe visually moved.

In the previous technique, color difference motion componentscorresponding to a temporal frequency of a moving image are required soas to present the moving image. The configuration is described belowwhich provides an illusion that an image is moving in a temporalfrequency which is larger than a predetermined level without using acolor difference of motion component having the temporal frequency whichis larger than a predetermined level.

Human beings detect local motion components from a moving image by usinga function like spatiotemporal filtering (Reference Literature 3:Watson, A. B., & Ahumada, A. J, “Model of human visual-motion sensing,”Journal of the Optical Society of America, (1985), A, 2, 322-342). Afunction similar to this can be realized by image information processingin which motion components and static components are separatelyextracted from a moving image through a temporal frequency filter or thelike.

It is known that human beings percept static image components, which aredefined by colors or luminance contrast, while drawing the static imagecomponents toward luminance motion components (“Reference Literature 4:Ramachandran, V. S., “Interaction between colour and motion in humanvision,” Nature (1987), 328, 645-647.” and “Reference Literature 5:Anstis, S., “Kinetic edges become displaced, segregated, and invisible,”In D. M.-K. Lam (Ed.), Neural mechanisms of visual perception,Proceedings of the Second Retina Research Foundation Conference, Texas(1989): Portfolio Press, 247-260”). From this visual characteristics, itis considered possible to modulate an appearance of a picture image byluminance motion components in a moving image.

The human visual system is insensitive to a motion signal which isdefined by a color signal, while the human visual system is sensitive toa motion signal which is defined by a luminance signal (ReferenceLiterature 6: Ramachandran, V. S., & Gregory, R. L, “Does colour providean input to human motion perception?” Nature (1978), 275, 55-56). Inaddition, spatial resolution of human beings with respect to an objectwhich moves at a predetermined or higher velocity is lower than thatwith respect to a static object (Reference Literature 7: Kelly, D. H.(1979). Motion and vision. II. Stabilized spatio-temporal thresholdsurface. Journal of the Optical Society of America, 69, 1340-1349). Thatis, even if color components are eliminated from motion components of amoving image, the quality of a video which is perceived by human beingsis not largely lowered.

In each embodiment, in the light of the above-mentioned perceptionalcharacteristics of human beings, a “luminance motion component having atemporal frequency an absolute value of which is equal to or larger thanthe second value” is put on an “image having a temporal frequency anabsolute value of which is equal to or smaller than the first value” andincluding a spatial frequency component an absolute value of which islarger than zero. Here, the “luminance motion component” is a componentcorresponding to the “image” and “the second value” is larger than “thefirst value”. “Including a spatial frequency component an absolute valueof which is larger than zero” represents including a spatial frequencycomponent which is not zero. For example, when it is assumed that “thefirst value” is F₁ and “the second value” is F₂, (0≤F₁<F₂ is satisfied.F₁=0 may be accepted or F₁>0 may be accepted. An example of F₁ is anabsolute value of a temporal frequency at which an “image” is perceivedto be standing (for example, approximately zero). Here, F₁ may be atemporal frequency at which the “image” is perceived to be moving. Inthe case of F₁=0, the “image having a temporal frequency an absolutevalue of which is equal to or smaller than the first value” is a staticimage. In the case of F₁>0, the “image having a temporal frequency anabsolute value of which is equal to or smaller than the first value” isan image which exhibits a motion having a frequency component themagnitude of which is equal to or smaller than the first value and movesmore slowly than a “luminance motion component having a temporalfrequency an absolute value of which is equal to or larger than thesecond value”. The “image” may have only a single temporal frequencycomponent (for example, 0 Hz) or may have several pieces of temporalfrequency components (that is, may be a composition of images of severalpieces of temporal frequencies). The “component having a temporalfrequency an absolute value of which is equal to or smaller than thefirst value” is referred to as a “low temporal frequency component” andthe “component having a temporal frequency an absolute value of which isequal to or larger than the second value” is referred to as a “hightemporal frequency component” below. It is preferable that the “image”is not uniform and includes a spatial frequency component an absolutevalue of which is larger than zero (a spatial frequency component whichis not zero). Both of a color difference component and a luminancecomponent of the “image” may include a spatial frequency component anabsolute value of which is larger than zero or one of the colordifference component and the luminance component may include a spatialfrequency component an absolute value of which is larger than zero. Ahuman being who watches a video obtained by putting a “luminance motioncomponent” on such “image” is given an illusion that the “image” ismoving at a higher temporal frequency than a temporal frequency havingthe magnitude of “the first value” (dynamic illusion). For example, inthe case where a “luminance motion component” is put on a static“image”, an illusion that this “image” is moving is given to humanbeings. Here, “a frequency α is higher than a frequency β” representsthat an absolute value |α| of the frequency α is larger than an absolutevalue IN of the frequency β.

The “image” may be an achromatic (grayscale) image including only aluminance component or may be an image contaming a chromatic color(color). Especially, in the case where a luminance motion component isput on the latter image, such illusion can be given that a chromaticimage is moving at a higher frequency than a low temporal frequencycomponent even though the chromatic image does not include a colordifference component higher than the low temporal frequency component.The “image” may be a picture image (image information) which is anobject of information processing or an image which is expressed on asurface of the object. Examples of the “image” which is expressed on asurface of an object include a picture image and a photograph which are“printed”, “drawn”, “displayed”, or “projected” on the surface of theobject, a pattern and a design based on a tone of color of materialsconstituting the surface of the object, a pattern based on a shape ofthe surface of the object (such as a design, a boundary line, and ashade), and the like. The “surface of the object” may be a flat surface,a curved surface, or an uneven surface. The “object” may be a thinghaving a three-dimensional shape (such as a vase, a ball, a model, and abuilding) or a thing which can be considered as a plane from theviewpoint of the application (such as a paper, a board, a wall, ascreen, a picture plane, and a transmission type display).

In order to provide a large dynamic illusion, it is preferable that an“image” and a “luminance motion component” correspond to an identical“moving image”. For example, an “image” corresponds to a componenthaving a temporal frequency which is zero (temporal frequency=0 Hz) orapproximately zero in “several pieces of frames” of a “moving image” anda “luminance motion component” corresponds to a luminance componenthaving a temporal frequency an absolute value of which is positive(|temporal frequency|>0 Hz) in the “plurality of pieces of frames”.Especially, in the case where a “moving image” includes a “periodic orrepetitive motion component”, the larger advantageous effect can beexpected. The “periodic motion component” represents not only acomponent for performing the precisely-periodic motion but also acomponent for performing a motion with high periodicity. In a similarmanner, the “repetitive motion component” represents not only acomponent for performing the precisely-repetitive motion but also acomponent for performing a highly-repetitive motion. In the case where a“moving image” includes a “periodic or repetitive motion component”, an“image” may correspond to a static image in an arbitrary frame includedin the “moving image”. “A corresponds to B” may represent that A is B, Ais derived from (is based on) B, or B is derived from A. “A is derivedfrom B” may represent that A is obtained from B, A is obtained from aduplicate of B, or A is obtained from an approximation of B. Forexample, an “image” and a “luminance motion component” may be extractedfrom a “moving image” or a duplication of the moving image or a “movingimage” may be generated from a static “image” which is taken by a cameraor a scanner so as to extract a “luminance motion component” from the“moving image”.

A function of motion vision is constant with respect to the luminancecontrast which is equal to or larger than a predetermined value(Reference Literature 8: Pantle, A., & Sekuler, R. (1969). Contrastresponse of human visual mechanisms sensitive to orientation anddirection of motion. Vision Research, 9, 397-406). Further, spatialresolution in motion view is lower than spatial resolution in perceptionwith respect to a static image (Reference Literature 7). That is, evenif spatial resolution and contrast of a luminance motion component whichis extracted from a moving image are controlled and thus the quality ofmotion information itself is degraded, the quality of a moving image tobe perceived is maintained. Accordingly, even if a component obtained byreducing a high spatial frequency component and contrast of a luminancemotion component moving image which is contained in the “moving image”is set to be a “luminance motion component”, a sufficient dynamicillusion can be created. Thus, the information amount can be reducedalmost without reducing the degree of a dynamic illusion.

“A luminance motion component is put on an image” represents that “aluminance motion component is synthesized with an image”, “a luminancemotion component is superimposed on an image”, “a luminance motioncomponent is integrated with an image”, “a luminance motion component isadded to an image”, “a luminance motion component is reflected on animage”, “a luminance motion component is incorporated into an image”, or“calculation including at least addition, multiplication, orexponentiation is applied to a pixel value of an image and a pixel valueof a luminance motion component”, for example. A specific method for“putting a luminance motion component on a picture image” will bedescribed later.

Fifth Embodiment

In the present embodiment, a “low temporal frequency component” and a“high temporal frequency component” are extracted from a moving imageM₁. A “low temporal frequency component” is defined as “a picture image(an image)” and a luminance component extracted from a “high temporalfrequency component” is defined as a “luminance motion component”. Acolor difference component of a “high temporal frequency component” isnot used. Calculation for integrating (putting) a “luminance motioncomponent” into (on) a “picture image” is performed and a moving imageM₂ obtained through the calculation is displayed (FIG. 13A).Accordingly, the “picture image” and the “luminance motion component”are integrated with each other in the visual system of a human being whowatches the moving image M₂ so as to create an illusion that the“picture image” is moving at a higher temporal frequency than the “lowtemporal frequency”. Thus, even though a color difference component ofthe “high temporal frequency component” is eliminated from the movingimage M₁, the quality of a video in perception is maintained.

<Configuration>

As illustrated in FIG. 15, a moving image component extraction device 51according to the present embodiment includes a low temporal frequencycomponent extractor 511 (first processor), a luminance motion componentextractor 512 (second processor), and an output unit 513. A dynamicillusion presentation device 52 (illusion presentation device) accordingto the present embodiment includes an input unit 521, an arithmetic unit522, and a display unit 523. The moving image component extractiondevice 51 and the low temporal frequency component extractor 511 aredevices which are configured when a predetermined program is read into ageneral-purpose or dedicated computer which is provided with a processor(hardware processor) such as a central processing unit (CPU), a memorysuch as a random-access memory (RAM) and a read-only memory (ROM), adisplay device such as a display, and the like. This computer may beprovided with a single processor or a single memory or provided withseveral processors and several memories. This program may be installedon the computer or may be recorded in the ROM or the like in advance. Apart or the whole processing unit may be configured not by an electroniccircuit (circuitry) which realizes the functional configuration when aprogram is read in such as a CPU but by an electronic circuit whichrealizes the processing function without using a program. Further, anelectronic circuit constituting a single piece of device may includeseveral CPUs.

<Processing>

Processing according to the present embodiment is described withreference to FIG. 16. A moving image M₁ according to the presentembodiment is a color moving image including a chromatic color and iscomposed of pixel values representing intensity change of the R channel,the G channel, and the B channel (Formula (1)).

$\begin{matrix}{M_{1} = \left\{ \begin{matrix}\left\{ {R\left( {x,y,t} \right)} \right\} \\\left\{ {G\left( {x,y,t} \right)} \right\} \\\left\{ {B\left( {x,y,t} \right)} \right\}\end{matrix} \right.} & (1)\end{matrix}$

where {R(x, y, t)}, {G(x, y, t)}, and {B(x, y, t)} are three-dimensionalmatrices including two-dimensional information for space andone-dimensional information for time which respectively have pixelvalues R(x, y, t), G(x, y, t), and B(x, y, t) as elements. The pixelvalues R(x, y, t), G(x, y, t), and B(x, y, t) respectively representsthe intensity of the R channel, the intensity of the G channel, and theintensity of the B channel on a horizontal position x, a verticalposition y, and a frame number t. x, y, and t are integers respectivelyrepresenting a horizontal position, a vertical position, and a framenumber in the case where a moving image is expressed by athree-dimensional coordinate system. A lower limit and an upper limit ofa horizontal position are respectively denoted as x_(min) and x_(max)(x_(min)<x_(max)), a lower limit and an upper limit of a verticalposition are set as y_(mm), and y_(max) (y_(min)<y_(max)), and a lowerlimit and an upper limit of a frame number are respectively denoted ast_(min) and t_(max) (t_(min)<t_(max)). x, y, and t satisfyx_(min)≤x≤_(max), y_(min)≤y≤_(max), and t_(min)≤t≤t_(max) respectively.It is preferable that the moving image M₁ has frames (non-uniformframes) having spatial frequency components an absolute value of whichis larger than zero (spatial frequency components which are not zero).

The pixel values R(x, y, t), G(x, y, t), and B(x, y, t) constituting themoving image M₁ (where x_(min)≤x≤x_(max), y_(min)≤y≤_(max), andt_(min)≤t≤t_(max)) are input into the low temporal frequency componentextractor 511 and the luminance motion component extractor 512 of themoving image component extraction device 51 (FIG. 15).

The low temporal frequency component extractor 511 acquires staticcomponents R_(static)(x, y), G_(static)(x, y), and B_(static)(x, y) fromthe pixel values R(x, y, t), G(x, y, t), and B(x, y, t) (wherex_(min)≤x≤x_(max), y_(min)≤y≤y_(max), and t_(min)≤t≤t_(max)) so as tooutput the static components R_(static)(x, y), G_(static)(x, y), andB_(static)(x, y). In the present embodiment, temporal average values ofthe pixel values R(x, y, t), G(x, y, t), and B(x, y, t) are set asR_(static)(x, y), G_(static)(x, y), and B_(static)(x, y) respectively(Formula (2)).

R _(static)(x,y)=Mean[R(x,y,t)]_(t) _(min) ≤t≤t _(max)

G _(static)(x,y)=Mean[G(x,y,t)]_(t) _(min) _(≤t≤t) _(max)

B _(static)(x,y)=Mean[B(x,y,t)]_(t) _(min) _(≤t≤t) _(max)   (2)

where Mean[x(t)]_(a≤t≤b) represents an average value of x(a), . . . ,x(b).

The low temporal frequency component extractor 511 outputs a staticimage M_(static) (Formula (3)) composed of two-dimensional matrices{R_(static)(x, y)}, {G_(static)(x, y)}, and {B_(static)(x, y)} whichhave R_(static)(x, y), G_(static)(x, y), and B_(static)(x, y) (wherex_(min)≤x≤x_(max) and y_(min)≤y≤_(max)) respectively as elements.

$\begin{matrix}{M_{static} = \left\{ \begin{matrix}\left\{ {R_{static}\left( {x,y} \right)} \right\} \\\left\{ {G_{static}\left( {x,y} \right)} \right\} \\\left\{ {B_{static}\left( {x,\ y} \right)} \right\}\end{matrix} \right.} & (3)\end{matrix}$

The image M_(static) is a component a temporal frequency of which iszero in several pieces of frames t (t_(min)≤t≤t_(max)) of the movingimage M₁ and is an example of an “image having a temporal frequency anabsolute value of which is equal to or smaller than the first value” inthe case where “the first value” is set to be 0. It is preferable thatthe image M_(static) static includes a spatial frequency component anabsolute value of which is larger than zero and contains a chromaticcolor (step S511).

The luminance motion component extractor 512 sets the pixel values R(x,y, t), G(x, y, t), and B(x, y, t) as R_(original)(x, y, t), andB_(original)(x, y, t) respectively (step S5121) and applies weightaddition to R_(original)(x, y, t), G_(original)(x, y, t), andB_(original)(x, y, t) depending on degrees, at which respective colorscontribute for luminance, so as to obtain a luminance componentY_(original)(x, y, t) (where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max), andt_(min)≤t≤_(max)) of the moving image M₁ (step S5122) (Formula (4)).Y_(original)(x, y, t)=α_(R)R_(original)(x, y, t)+α_(G)G_(original)(x, y,t)+α_(B)B_(original)(x, y, t) (4) where α_(R), α_(G), and α_(B) areweight coefficients (constants) (for example, α_(R)=0.299, α_(G)=0.587,and α_(B)=0.114).

Further, the luminance motion component extractor 512 subtracts theluminance static component Y_(static)(x, y) from the luminance componentY_(original)(x, y, t) of each frame t so as to obtain and output theluminance component Y_(static)(x, y) is obtained by time-averaging theluminance component Y_(original)(x, y, t) (where x_(min)≤x≤x_(max),y_(min)≤y≤y_(max), and t_(min)≤t≤t_(max)).

Y _(motion)(x,y,t)=Y _(original)(x,y,t)−Y _(static)(x,y)  (5)

The luminance motion component Y_(motion)(x, y, t) is a luminancecomponent having a temporal frequency an absolute value of which ispositive in several pieces of frames t (t_(min)≤t≤t_(max)) of the movingimage M₁ and is an example of a “luminance motion component having atemporal frequency an absolute value of which is equal to or larger thanthe second value”. This luminance motion component is a componentcorresponding to an image and the second value is larger than the firstvalue (step S5123).

The image M_(static) and the luminance motion component Y_(motion)(x, y,t) (where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max), and t_(min)≤t≤t_(max)),and are input into the output unit 513 to be transferred to the dynamicillusion presentation device 52. The image M_(static) and the luminancemotion component Y_(motion)(x, y, t) are input into the input unit 521of the dynamic illusion presentation device 52 to be transferred to thearithmetic unit 522. The arithmetic unit 522 acquires and outputs amoving image M₂ obtained by putting the luminance motion componenty_(motion)(x, y, t) on the image M_(static). For example, the arithmeticunit 522 adds the luminance motion component Y_(motion)(x, y, t) to eachof the static components R_(static)(x, y), G_(static)(x, y), andB_(static)(x, y) (where x_(min)≤x≤x_(max) and y_(min)≤y≤y_(max)) of theimage M_(static) so as to obtain the moving image M₂ (FIG. 18A, Formula(6))

$\begin{matrix}{M_{2} = \left\{ \begin{matrix}{\left\{ {R_{motion}^{\prime}\left( {x,y,t} \right)} \right\} = \left\{ {{R_{static}\left( {x,y} \right)} + {Y_{motion}^{\prime}\left( {x,y,t} \right)}} \right\}} \\{\left\{ {G_{motion}^{\prime}\left( {x,y,t} \right)} \right\} = \left\{ {{G_{static}\left( {x,y} \right)} + {Y_{motion}^{\prime}\left( {x,y,t} \right)}} \right\}} \\{\left\{ {B_{motion}^{\prime}\left( {x,y,t} \right)} \right\} = \left\{ {{B_{static}\left( {x,y} \right)} + {Y_{motion}^{\prime}\left( {x,y,t} \right)}} \right\}}\end{matrix} \right.} & (6)\end{matrix}$

Accordingly, the moving image M₂ obtained by eliminating color motioninformation from the moving image M₁ can be generated (step S522). Here,in the case of Formula (6), the chromaticity (a ratio of RGB) of eachpixel of the moving image M₂ slightly varies from the moving image M₁(original moving image). However, the human visual characteristics aregenerally insensitive to color change and the apparent quality of themoving image M₂ does not deteriorate compared to the moving image M₁.When the chromaticity of each pixel is desired to be saved, each of theRGB channels may be multiplied by the modulation ratio Y′_(motion)(x, y,t) based on a motion of the luminance static component Y_(static)(x, y)(Formula (7), FIG. 18B).

$\begin{matrix}{M_{2} = \left\{ {{\begin{matrix}{\left\{ {R_{motion}^{\prime}\left( {x,y,t} \right)} \right\} = \left\{ {{R_{static}\left( {x,y} \right)} + {Y_{motion}^{\prime}\left( {x,y,t} \right)}} \right\}} \\{\left\{ {G_{motion}^{\prime}\left( {x,y,t} \right)} \right\} = \left\{ {{G_{static}\left( {x,y} \right)} + {Y_{motion}^{\prime}\left( {x,y,t} \right)}} \right\}} \\{\left\{ {B_{motion}^{\prime}\left( {x,y,t} \right)} \right\} = \left\{ {{B_{static}\left( {x,y} \right)} + {Y_{motion}^{\prime}\left( {x,y,t} \right)}} \right\}}\end{matrix}{Y_{motion}^{\prime}\left( {x,y,t} \right)}} = \frac{{Y_{static}\left( {x,y} \right)} + {Y_{motion}\left( {x,y,t} \right)}}{Y_{static}\left( {x,y} \right)}} \right.} & (7)\end{matrix}$

In this case, the ratio among the intensity of the RGB channels ismaintained, so that a luminance motion component without modulation of acolor signal can be synthesized (step S522). That is, in the presentembodiment, a video (Y_(motion)(x, y, t) or Y′_(motion)(x, y, t)) issuperimposed on an object in order that the object (M_(static)) will beperceived as if the object (M_(static)) were given a motion. This videois a video including a luminance motion component corresponding to amotion given to the object (for example, a video including only aluminance component). Further, as illustrated in Formula (6) and Formula(7), the video is superimposed on the object so that a region of theobject (M_(static)) and a region, which corresponds to a motion given tothe object, in the video (Y_(motion)(x, y, t) or Y′_(motion)(x, y, t))are overlapped with each other. Further, an absolute value of a temporalfrequency of the luminance motion component in several pieces of framesin the video is larger than an absolute value of a temporal frequency inseveral pieces of frames in the video corresponding to the object.

The moving image M₂ thus obtained is input into the display unit 523 soas to be displayed from there (step S523). Even though the moving imageM₂ does not include a motion component of a color signal, an illusion ofa motion is provided. That is, even though a color motion component iseliminated from the moving image M₁, a visual experience comparable tothe original moving image can be provided to a user.

[Modification 1 of Fifth Embodiment]

The processing for separating and extracting each component from themoving image M₁ can be realized also by converting the moving image M₁into a temporal frequency region by Fourier transform or the like andperforming temporal frequency filtering. Hereinafter, differences withthe above-described matters will be focused and described. Descriptionsof the matters which have already been described will be sometimesomitted by using the same reference characters.

<Configuration>

As illustrated in FIG. 15, a moving image component extraction device51′ according to the present embodiment includes a frequency regionconverter 514′, a low temporal frequency component extractor 511′ (firstprocessor), a high temporal frequency component extractor 515′, aluminance motion component extractor 512′ (second processor), temporalregion converters 516′ and 517′, and an output unit 513. The dynamicillusion presentation device 52 is identical to that of the fifthembodiment. The moving image component extraction device 51′ isconfigured when a predetermined program is read into a computer as thatdescribed above, for example.

<Processing>

Processing according to the present modification is described withreference to FIG. 17. The above-described pixel values R(x, y, t), G(x,y, t), and B(x, y, t) constituting the moving image M₁ (wherex_(min)≤x≤x_(max), y_(min)≤y≤_(max), and t_(min)≤t≤_(max)) are inputinto the frequency region converter 514′. The frequency region converter514′ converts the pixel values R(x, y, t), G(x, y, t), and B(x, y, t)into temporal frequency regions so as to obtain values FR(x, y, f),FG(x, y, f), and FB(x, y, f) of the temporal frequency regions. Anexample in which Fourier transform is used is described below (Formula(8)).

FR(x,y,f)=∫R(x,y,t)exp(−2πitf)dt

FG(x,y,f)=∫G(x,y,t)exp(−27πitf)dt

FB(x,y,f)=∫B(x,y,t)exp(−2πitf)dt  (8)

In this case, FR(x, y, f), FG(x, y, f), and FB(x, y, f) are Fourierspectra on the dimension t corresponding to R(x, y, t), G(x, y, t), andB(x, y, t) (where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max), andt_(min)≤t≤_(max)). Though the integration range of Formula (8) is from−∞ to +∞, calculation may be actually performed only with respect to alimited interval t_(min)≤t≤t_(max). Further, R(x, y, t), G(x, y, t), andB(x, y, t) are discrete values, so that discrete Fourier transform maybe used. f denotes an integer index representing a temporal frequency.f=0 represents that a temporal frequency is 0 Hz and larger f representsa higher temporal frequency (temporal frequency an absolute value ofwhich is larger). When an upper limit of |f| is denoted as f_(max),0≤|f|≤f_(max) is satisfied. Further, i denotes an imaginary unit and πdenotes a circumference ratio. FR(x, y, f), FG(x, y, f), and FB(x, y, f)(where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max), and 0≤|f|≤f_(max)) aretransferred to the low temporal frequency component extractor 511′ andthe high temporal frequency component extractor 515′ (step S514′).

The low temporal frequency component extractor 511′ multiplies FR(x, y,f), FG(x, y, f), and FB(x, y, f) by a low-pass filter LF(f) so as toobtain and output GR_(static)(x, y, f), GG_(static)(x, y, f), andGB_(static)(x, y, f) (Formula (9)).

GR _(static)(x,y,f)=FR(x,y,f)LF(f)

GG _(static)(x,y,f)=FG(x,y,f)LF(f)

GB _(static)(x,y,f)=FB(x,y,f)LF(f)  (9)

where the low-pass filter LF(f) is expressed by Formula (10).

$\begin{matrix}{{{LF}(f)} = \left\{ \begin{matrix}1 & {{{for}\mspace{14mu}{f}} \leq k} \\0 & {otherwise}\end{matrix} \right.} & (10)\end{matrix}$

where k denotes a constant which is equal to or larger than 0. In thecase of k=0, the low-pass filter LF(f) extracts a component having atemporal frequency of 0 Hz. In the case of k>0, the low-pass filterLF(f) extracts a component having a temporal frequency an absolute valueof which is equal to or smaller than that of a temporal frequencycorresponding to k (step S511′).

GR_(static)(x, y, f), GG_(static)(x, y, f), and GB_(static)(x, y, f) areinput into the temporal region converter 516′. The temporal regionconverter 516′ converts GR_(static)(x, y, f), GG_(static)(x, y, f), andGB_(static)(x, y, f) into temporal regions so as to obtain low temporalfrequency components R_(static)(x, y, t), G_(static)(x, y, t), andB_(static)(x, y, t). An example in which inverse Fourier transform isused is described below (Formula (11)).

R _(static)(x,y,t)=∫GR _(static)(x,y,f)exp(2πitf)df

G _(static)(x,y,t)=∫GG _(static)(x,y,f)exp(2πitf)df

B _(static)(x,y,t)=∫GB _(static)(x,y,f)exp(2πitf)df  (11)

Though the integration range of Formula (11) is from −∞ to +Go,calculation may be actually performed only with respect to a limitedinterval 0≤|f|≤f_(max). Further, inverse discrete Fourier transform maybe used. The temporal region converter 516′ outputs a picture image(image) M_(static) including low temporal frequency components below.

$\begin{matrix}{M_{static} = \left\{ \begin{matrix}\left\{ {R_{static}\left( {x,y,t} \right)} \right\} \\\left\{ {G_{static}\left( {x,y,t} \right)} \right\} \\\left\{ {B_{static}\left( {x,\ y,t} \right)} \right\}\end{matrix} \right.} & (12)\end{matrix}$

This image M_(static) is an example of an “image having a temporalfrequency an absolute value of which is equal to or smaller than thefirst value”. In the case where k=0 is set in Formula (10),R_(static)(x, y, t), G_(static)(x, y, t), and B_(static)(x, y, t) arerespectively temporal average values of R(x, y, t), G(x, y, t), and B(x,y, t) and the M_(static) is a static picture image. The image M_(static)in this case is composed of components a temporal frequency of which iszero in several pieces of frames (t_(min)≤t≤_(max)) of the moving imageM₁. In the case where k>0 is set in Formula (10), M_(static) is apicture image which includes components of a slow action (step S516′).

The high temporal frequency component extractor 515′ multiplies FR(x, y,f), FG(x, y, f), and FB(x, y, f) by a high-pass filter HF(f) so as toobtain and output GR_(motion)(x, y, f), GG_(motion)(x, y, f), andGB_(motion)(x, y, f) (where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max), and0≤|f|≤f_(max)) (Formula (13)).

GR _(motion)(x,y,f)=FR(x,y,f)HF(f)

GG _(motion)(x,y,f)=FG(x,y,f)HF(f)

GB _(motion)(x,y,f)=FB(x,y,f)HF(f)  (13)

where the high-pass filter HF(f) is expressed by Formula (14).

$\begin{matrix}{{H\;{F(f)}} = \left\{ \begin{matrix}0 & {{{for}\mspace{14mu}{f}} \leq h} \\1 & {otherwise}\end{matrix} \right.} & (14)\end{matrix}$

h denotes a constant which is equal to or larger than 0. In the case ofh=0, the high-pass filter HF(f) extracts a component having a temporalfrequency an absolute value of which is positive. In the case of h>0,the high-pass filter HF(f) extracts a temporal frequency componenthaving a temporal frequency an absolute value of which is larger thanthat of a temporal frequency corresponding to h. It is preferable that his equal to or larger than k. k and h do not necessarily have to beequal to each other. In the case where k and h are set to be equal toeach other, the low-pass filter LF(f) and the high-pass filter HF(f) arecomplementary to each other. In this case, instead of the use of thehigh-pass filter HF(f), components which are not removed in the lowtemporal frequency component extractor 511′ may be set to beGR_(motion)(x, y, f), GG_(motion)(x, y, f), and GB_(motion)(x, y, f).That is,

GR _(motion)(x,y,f)=FR(x,y,f)−GR _(static)(x,y,f)

GG _(motion)(x,y,f)=FG(x,y,f)−GG _(static)(x,y,f)

GB _(motion)(x,y,f)=FB(x,y,f)−GB _(static)(x,y,f)  (15)

may be set (step S515′).

GR_(motion)(x, y, f), GG_(motion)(x, y, f), and GB_(motion)(x, y, f) areinput into the temporal region converter 517′. The temporal regionconverter 517′ converts GR_(motion)(x, y, f), GG_(motion)(x, y, f), andGB_(motion)(x, y, f) into temporal regions so as to obtain and outputhigh temporal frequency components R_(motion)(x, y, t), G_(motion)(x, y,t), and B_(motion)(x, y, t) (where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max),and t_(min)≤t≤t_(max)). An example in which inverse Fourier transform isused is described below (Formula (16)).

R _(motion)(x,y,t)=∫GR _(motion)(x,y,f)exp(2πitf)df

G _(motion)(x,y,t)=∫GG _(motion)(x,y,f)exp(2πitf)df

B _(motion)(x,y,t)=∫GB _(motion)(x,y,f)exp(2πitf)df  (16)

Though the integration range of Formula (16) is from −∞ to +∞,calculation may be actually performed only with respect to a limitedinterval 0≤|f|≤f_(max). Further, inverse discrete Fourier transform maybe used (step S517′).

The luminance motion component extractor 512′ applies weight addition toR_(motion)(x, y, t), G_(motion)(x, y, t), and B_(motion)(x, y, t) so asto obtain and output the luminance component Y_(original)(x, y, t)(where x_(m)≤x≤x_(max), y_(min)≤y≤y_(max), and t_(min)≤t≤_(max)) of themoving image M₁ (step S512′) (Formula (17)).

Y _(motion)(x,y,t)=α_(R) R _(motion)(x,y,t)+α_(G) G_(motion)(x,y,t)+α_(B) B _(motion)(x,y,t)  (17)

where α_(R), α_(G), and α_(B) are weight coefficients (constants). Theluminance motion component Y_(motion)(x, y, t) is an example of a“luminance motion component having a temporal frequency an absolutevalue of which is equal to or larger than the second value”.

The following processing is same as that of the fifth embodiment. In thepresent embodiment, an illusion that the moving image M₂ is moving atthe higher temporal frequency than the image M_(static) can be providedto a user.

[Modification 2 of Fifth Embodiment]

In the fifth embodiment and modification 1 of the fifth embodiment, themoving image M₂ is generated which is obtained by putting a luminancemotion component extracted from a high temporal frequency component ofthe moving image M₁ on the image M_(static) composed of a low temporalfrequency component of the moving image M₁ (FIG. 13A). However, themoving image M₂ may be generated which is obtained by putting aluminance motion component extracted from a high temporal frequencycomponent of the moving image M₁ on the image M_(static) of a colorcomponent which is defined based on a “color difference component”extracted from a low temporal frequency component of the moving image M₁(FIG. 13B). Alternatively, the moving image M₂ may be generated which isobtained by putting a luminance motion component extracted from a hightemporal frequency component of the moving image M₁ on the imageM_(static) composed of a “luminance component” extracted from a lowtemporal frequency component of the moving image M₁ (FIG. 14A). Evensuch moving image M₂ can provide a dynamic illusion. Further, the movingimage M₁ may be a grayscale moving image.

Sixth Embodiment

In the case where the moving image M₁ includes a periodical orrepetitive motion component (for example, a small action), a staticimage in an arbitrary frame (for example, the t=n-th frame) included inthe moving image M₁ may be used as a “picture image” (FIG. 13A). Thatis, the moving image M₂ may be generated which is obtained by putting aluminance motion component on a static image of an arbitrary frameextracted from the moving image M₁. An illusion that a static image islooked moving can be provided by such moving image M₂ as well.

<Configuration>

As illustrated in FIG. 15, a moving image component extraction device 61according to the present embodiment includes an image extractor 611(first processor), a luminance motion component extractor 612 (secondprocessor), and an output unit 513. The dynamic illusion presentationdevice 52 is identical to that of the fifth embodiment. The moving imagecomponent extraction device 61 is configured when a predeterminedprogram is read into a computer as that described above, for example.

<Processing>

Processing according to the present embodiment is described withreference to FIG. 16. The difference from the fifth embodiment is thatstep S611 is executed instead of step S511 and step S6123 is executedinstead of step S5123. Hereinafter, only step S611 and step S6123 whichare the differences will be described.

«Step S611»

The moving image M₁ (Formula (1)) is input into the image extractor 611and the luminance motion component extractor 612. The image extractor611 extracts a static image of the t=n-th frame from the moving image M₁and outputs the static image as the image M_(static). That is, the imageM_(static) (Formula (3)) composed of two-dimensional matrices{R_(static)(x, y)}, {G_(static)(x, y)}, and {B_(static)(x, y)} whichhave R_(static)(x, y)=R(x, y, n), G_(static)(x, y)=G(x, y, n), andB_(static)(x, y)=B(x, y, n) (x_(min)≤x≤x_(max) and y_(min)≤y≤y_(max))respectively as elements is outputted. The image M_(static) of thepresent embodiment is also an example of an “image having a temporalfrequency an absolute value of which is equal to or smaller than thefirst value” in the case where “the first value” is set to be 0. It ispreferable that the image M_(static) includes a spatial frequencycomponent an absolute value of which is larger than zero and contains achromatic color (step S611).

«Step S6123»

The luminance motion component extractor 612 sets luminance componentscorresponding to R_(static)(x, y), G_(static)(x, y), and B_(static)(x,y) (where x_(min)≤x≤x_(max) and y_(min)≤y≤y_(max)) to be the luminancestatic component y_(static)(x, y). The luminance static componentY_(static)(x, y) is obtained by applying weight addition toR_(static)(x, y), G_(static)(x, y), and B_(static)(x, y) as is the casewith step S5122. The luminance motion component extractor 612 subtractsthe luminance static component Y_(static)(x, y) from the luminancecomponent Y_(original)(x, y, t) of each frame t so as to obtain andoutput the luminance motion component Y_(motion)(x, y, t) (wherex_(min)≤x≤x_(max), y_(min)≤y≤y_(max), and t_(min)≤t≤t_(max)) (stepS6123) (Formula (5)). That is, a video (Y_(motion)(x, y, t)) of thepresent embodiment includes a luminance motion component which isobtained by subtracting the luminance static component Y_(static)(x, y),which is obtained from a static image based on a video corresponding toan object, from the luminance component y_(original)(x, y, t) of each ofseveral pieces of frames in the video corresponding to the object.

In the present embodiment, an illusion that a single static image of anarbitrary frame is visually moving can be provided by a luminance motioncomponent and a visual experience comparable to the original movingimage can be provided to a user.

[Modification of Sixth Embodiment]

In the present embodiment, the moving image M₂ is generated which isobtained by putting a luminance motion component extracted from a hightemporal frequency component of the moving image M₁ on the static imageM_(static) of an arbitrary frame of the moving image M₁ (FIG. 13A).However, the moving image M₂ may be generated which is obtained byputting a luminance motion component extracted from a high temporalfrequency component of the moving image M₁ on a picture image of a colorcomponent which is defined based on a “color difference component”extracted from the static image M_(static) of an arbitrary frame of themoving image M₁ (FIG. 13B). Alternatively, the moving image M₂ may begenerated which is obtained by putting a luminance motion componentextracted from a high temporal frequency component of the moving imageM₁ on a picture image composed of a “luminance component” extracted fromthe static image M_(static) of an arbitrary frame of the moving image M₁(FIG. 14A). Even such moving image M₂ can provide a dynamic illusion.Further, the moving image M₁ may be a grayscale moving image.

Seventh Embodiment

As described above, due to the characteristics of motion view of thevisual system, even if spatial resolution or contrast of a “luminancemotion component” which is synthesized with a “picture image” isreduced, the quality of a moving image to be perceived is maintained. Inthe present embodiment, a “low temporal frequency component” of themoving image M₁ is defined as “a picture image (an image)” and acomponent obtained by reducing at least one of a high spatial frequencycomponent and contrast of a luminance component extracted from a “hightemporal frequency component” of the moving image M₁ (a luminance motioncomponent image included in the moving image M₁) is defined as a“luminance motion component”. For example, filtering for reducing atleast one of a high spatial frequency component and contrast isperformed with respect to a luminance motion component image.Subsequently, calculation for integrating (putting) the “luminancemotion component” into (on) the “picture image” is performed and themoving image M₂ obtained through the calculation is displayed (FIG.14B).

<Configuration>

As illustrated in FIG. 15, a moving image component extraction device 71according to the present embodiment includes a low temporal frequencycomponent extractor 511, a luminance motion component extractor 512, afiltering unit 719, and an output unit 513. The dynamic illusionpresentation device 52 of the present embodiment is identical to that ofthe fifth embodiment. The moving image component extraction device 71 isconfigured when a predetermined program is read into a computer as thatdescribed above, for example.

<Processing>

Processing according to the present embodiment is described withreference to FIG. 16. The difference from the fifth embodiment is thatfiltering is performed with respect to the luminance motion componenty_(motion)(x, y, t) (luminance motion component image) obtained in stepS5123 and the luminance motion component after the filtering is put onthe “picture image” so as to obtain the moving image M₂. Hereinafter,only processing of filtering with respect to the luminance motioncomponent y_(motion)(x, y, t) obtained in S5123 is described.

The luminance motion component Y_(motion)(x, y, t) obtained in stepS5123 (where x_(min)≤x≤x_(max), y_(min)≤y≤y_(max), andt_(min)≤t≤t_(max)) is input into the filtering unit 719. The filteringunit 719 first converts Y_(motion)(x, y, t) into a space-time frequencyregion so as to obtain FY_(motion)(ξ, η, τ). Here, ξ, η, and τrespectively denote a spatial frequency in the horizontal direction, aspatial frequency in the vertical direction, and a temporal frequency. Alower limit and an upper limit of the spatial frequency in thehorizontal direction are denoted as ξ_(min) and ξ_(max)(ξ_(min)<ξ_(max)) respectively, a lower limit and an upper limit of thespatial frequency in the vertical direction are set as η_(min) andη_(max) (η_(min)≤η_(max)), and a lower limit and an upper limit of thetemporal frequency are denoted as τ_(min) and τ_(max) (t_(min)<t_(max))respectively. ξ, η, and τ satisfy ξ_(min)≤ξ≤ξ_(max), η_(min)≤η≤η_(max),and τ_(min)≤τ≤τ_(max) respectively. An example in which Fouriertransform is used is described below (Formula (18)). FY_(motion)(ξ, η,τ) in this case is a Fourier spectrum of Y_(motion)(x, y, t).

FY _(motion)(ξ,η,τ)=∫∫∫Y_(motion)(x,y,t)exp[−2πi(x+ξ+yη+tτ)]dxdydt  (18)

Though the integration range of Formula (18) is from −∞ to +∞,calculation may be actually performed only with respect to limitedintervals x_(min)≤x≤_(max), y_(min)≤y≤y_(max), and t_(min)≤t≤_(max).Further, discrete Fourier transform may be used (step S7191).

Subsequently, the filtering unit 719 multiplies FY_(motion)(ξ, η, τ) bya filter G(ξ, η, τ) and further applies inverse Fourier transform so asto obtain a luminance motion component gY_(motion)(x, y, t) (Formula(19)).

gY _(motion)(x,y,t)=∫∫∫FY_(motion)(ξ,η,τ)G(ξ,η,τ)×exp[2πi(xξ+yη+tτ)]dξdηdτ  (19).

Though the integration range of Formula (19) is from −∞ to +∞,calculation may be actually performed only with respect to limitedintervals ξ_(min)≤ξ≤ξ_(max), η_(min)≤η≤η_(max), and τ_(min)≤τ≤τ_(max).Further, inverse discrete Fourier transform may be used. The luminancemotion component gY_(motion)(x, y, t) is an example of a “luminancemotion component having a temporal frequency an absolute value of whichis equal to or larger than the second value”.

G(ξ, η, τ) denotes a filter for reducing a high spatial frequencycomponent or contrast. A filter for reducing a high spatial frequencycomponent is a low-pass filter and a filter for reducing contrast (wholecontrast) is a function for linearly-converting a gray level, a functionfor planarizing a histogram (spatiotemporal frequency filter), or thelike, for example. A specific example of the low-pass filter is shownbelow (Formula (20)).

$\begin{matrix}{{G\left( {\xi,\eta,\tau} \right)} = \left\{ \begin{matrix}1 & {{{{for}\ {\xi }} \leq a},{{\eta } \leq b}} \\0 & {otherwise}\end{matrix} \right.} & (20)\end{matrix}$

where a and b denote positive constants. Though the function for cuttinga high spatial frequency Fourier spectrum in a stepwise manner isdenoted as G(ξ, η, τ), any functions may be denoted as G(ξ, η, τ) aslong as a high spatial frequency Fourier spectrum can be cut by thefunction (step S7192).

The following processing is same as processing in which the luminancemotion component Y_(motion)(x, y, t) of the fifth embodiment is replacedwith the luminance motion component gY_(motion)(x, y, t). As describedabove, even if a high spatial frequency of a luminance motion componentmoving image is reduced or contrast is reduced by filtering, the qualityof a moving image in perception is not affected within an allowablerange of the characteristics of the visual system. Therefore, even ifthe information amount of the luminance motion component is reduced byfiltering, a visual experience comparable to the original moving imageM₁ can be provided to a user.

[Modification of Seventh Embodiment]

The seventh embodiment is the embodiment in which the luminance motioncomponent Y_(motion)(x, y, t) of the fifth embodiment is replaced withthe luminance motion component gY_(motion)(x, y, t). However, theluminance motion component Y_(motion)(x, y, t) of modification 1 of thefifth embodiment may be replaced with the luminance motion componentgY_(motion)(x, y, t). In this case, steps S7191 and S7192 are executedafter step S512′ of FIG. 17. In a similar manner, the luminance motioncomponent Y_(motion)(x, y, t) of modification 2 of the fifth embodimentor the sixth embodiment may be replaced with the luminance motioncomponent gy_(motion)(x, y, t).

Eighth Embodiment

In an eighth embodiment, the above-described “image having a temporalfrequency an absolute value of which is equal to or smaller than thefirst value” appears on a surface of an “object” and a “luminance motioncomponent having a temporal frequency an absolute value of which isequal to or larger than the second value” is superimposed on this“image”. This can also provide an illusion that the “image” is moving.Hereinafter, an example will be described in which a static “image” isprinted on an “object” and a “luminance motion component having atemporal frequency an absolute value of which is equal to or larger thanthe second value” is projected on the “image”.

<Configuration>

As illustrated in FIG. 19, a moving image component extraction device 81according to the present embodiment is a device in which the output unit513 of any of the moving image component extraction devices 51, 51′, 61,and 71 according to the fifth embodiment to the seventh embodiment andmodifications of these embodiments is replaced with an output unit 813and a printing unit 814. A dynamic illusion presentation device 82(illusion presentation device) according to the present embodimentincludes an input unit 821 and a projection unit 823 (projector).Further, an object 83 illustrated in the present embodiment is atwo-dimensional medium such as a paper.

<Processing>

As any of the above-described embodiments and modifications, the staticimage M_(static) (Formula (3)) extracted from the moving image M₁ isinput into the printing unit 814 (an output unit). The printing unit 814prints the image M_(static) on a surface of the object 83. As any of theabove-described embodiments and modifications, the luminance motioncomponent Y_(motion)(x, y, t) or gY_(motion)(x, y, t) extracted from themoving image M₁ is transferred to the dynamic illusion presentationdevice 82 and input into the input unit 821. The luminance motioncomponent Y_(motion)(x, y, t) or gY_(motion)(x, y, t) is transferred tothe projection unit 823 and the projection unit 823 projects theluminance motion component Y_(motion)(x, y, t) or gY_(motion)(x, y, t)onto the image M_(static) printed on the object 83 by a known lightproduction technique (for example, Reference Literature 9: T. Kawabe, M.Sawayama, K. Maruya, S. Nishida, (2014), “A light projection method toperceptually deform two-dimensional static objects by motioninformation”, Annual conference of the Institute of Image Informationand Television Engineers 2014, 5-3) so as to display the moving image M₂(Formula (21)).

M ₂ =M _(static) ◯Y _(motion)(x,y,t)

or

M ₂ =M _(static) ◯gY _(motion)(x,y,t)  (21)

where ◯ of Formula (21) denotes a state in which addition andmultiplication of the luminance motion component Y_(motion)(x, y, t) orgY_(motion)(x, y, t) are compositely performed with respect to aluminance component of the image M_(static) (a putting state). In otherwords, such state is denoted that calculation including at least one ofaddition and multiplication is performed with respect to a luminancecomponent of the image M_(static) and the luminance motion componentY_(motion)(x, y, t) or gY_(motion)(x, y, t). That is, in the case wherelight is projected onto a printed material, it is presumed thatluminance changes multiplicatively in one part and luminance changesadditively in another part due to different reflection patternscorresponding to the characteristics of a paper or an ink. Therefore,calculation generating both of these luminance changes is denoted by ◯.Here, Formula (6) represents a state in which the luminance motioncomponent Y_(motion)(x, y, t) is added to a luminance component in theimage M_(static) and Formula (7) represents a stale in which a luminancecomponent in the image M_(static) is multiplied by a modulation ratioY′_(motion)(x, y, t) based on a motion of the luminance motion componentY_(motion)(x, y, t). That is, in the present embodiment, in order thatan object (M_(static)) will be perceived as if the object (M_(static))were given a motion, a video (Y_(motion)(x, y, t) or gY_(motion)(x, y,t)) is superimposed on the object. This video is a video including aluminance motion component corresponding to a motion given to the object(for example, a video including only a luminance component). Further, asillustrated in Formula (3) and Formula (21), the video is superimposedon the object so that a region of the object (M_(static)) and a region,which corresponds to a motion given to the object, in the video(Y_(motion)(x, y, t) or gY_(motion)(x, y, t)) are overlapped with eachother. Further, an absolute value of a temporal frequency of a luminancemotion component in several pieces of frames in the video is larger thanan absolute value of a temporal frequency in several pieces of frames inthe video corresponding to the object.

A dynamic illusion can be presented by such way as well. Here, in thecase where the luminance motion component y_(motion)(x, y, t) orgY_(motion)(x, y, t) is projected by a projector, the luminance motioncomponent Y_(motion)(x, y, t) or gY_(motion)(x, y, t) cannot have anegative value. Therefore, the whole luminance of the moving image M₂ isincreased. Further, the luminance increase by the projection partiallygoes in a multiplicative fashion. Therefore, the luminance distributionin the moving image M₂ is largely different from the luminance of theoriginal moving image M₁. Nevertheless, a user can have a visualexperience of a motion of the original moving image M₁ from the movingimage M₂. It is conceivable that adaptive luminance contrastnormalization by the visual system is related to this as described inReference Literature 9. Thus, a visual experience comparable to theoriginal moving image M₁ can be provided to a user in this embodiment aswell.

[Modification of Eighth Embodiment]

In the eighth embodiment, a static image M_(static) is printed on an“object” such as a paper. However, this may projected and displayed onan “object” such as a screen by another projector or may be displayed onan “object” such as an electronic paper, instead of printing this.Further, a “motion component moving image” may be projected on atransmission type display as that illustrated in Reference Literature 9.Further, in the case where the image M_(static) is projected ordisplayed, the image M_(static) does not have to be a static pictureimage but may be a picture image which moves slowly. In this case, avisual experience of a motion performed at a temporal frequency higherthan that of the image M_(static) can be provided to a user. Further,instead of extraction of the image M_(static) from the moving image M₁,the moving image M₁ may be produced based on a static image obtained byphotographing an “object” such as a building and a painting so as togenerate the luminance motion component Y_(motion)(x, y, t) orgY_(motion)(x, y, t) from the moving image M₁ or the luminance motioncomponent Y_(motion)(x, y, t) or gY_(motion)(x, y, t) may be generatedfrom a static image obtained by photographing an “object”. Thus, thegenerated luminance motion component Y_(motion)(x, y, t) orgY_(motion)(x, y, t) is projected onto an image appearing on a surfaceof an “object”, thereby being able to provide an illusion that the“image” is moving.

[Other Modifications Etc.]

Here, the present invention is not limited to the above-describedembodiments and modifications of these embodiments. For example, themoving image component extraction device and the dynamic illusionpresentation device may be identical devices. Alternatively, processingof each unit included in the moving image component extraction deviceand the dynamic illusion presentation device may be implemented bydevices different from each other.

The above-described various types of processing may be executed not onlyin a time-series manner in accordance with the description but also in aparallel manner or an independent manner, depending on processingcapability of a device which executes the processing or as necessary.Further, it is indisputable that alterations can be arbitrarily madewithout departing from the intent of the present invention.

In a case where the above-described configuration is realized by acomputer, processing contents of functions which should be obtained byrespective devices are described by a program. By executing this programby a computer, the above-described processing functions are realized onthe computer. The program in which the processing contents are describedcan be recorded in a recording medium which is readable by a computer.An example of the recording medium which is readable by a computer is anon-transitory recording medium. Examples of such recording mediuminclude a magnetic recording device, an optical disk, a magnetoopticalrecording medium, a semiconductor memory, and the like.

This program is distributed by selling, transferring, or lending aportable recording medium such as a DVD and a CD-ROM in which theprogram is recorded, for example. Further, this program may bedistributed such that this program is stored in a storage device of aserver computer and is transferred from the server computer to othercomputers through the network.

A computer which executes such program once stores the program which isrecorded in a portable recording medium or the program transferred fromthe server computer in a storage device thereof, for example. Inexecution of processing, this computer reads the program which is storedin a recording device thereof and executes processing in accordance withthe program which is read. As another execution form of this program, acomputer may directly read the program from a portable recording mediumso as to execute processing in accordance with the program, and further,the computer may sequentially execute processing in accordance with areceived program whenever a program is transferred from the servercomputer to this computer. The above-described processing may beexecuted by a service of an application service provider (ASP) type inwhich a processing function is realized only by an executing instructionof processing and result acquisition, without transferring the programto the computer from the server computer.

The data structure may be distributed in which “first picture image datafor recording a first picture image obtained by photographing” and“second picture image data for recording a second picture image which isa narrow-band image, the narrow-band image being obtained such that aspatial frequency band of a whole picture image is narrowed to benarrower than a spatial frequency band of the first picture image whilemaintaining information of an edge included in the first picture image,which is superimposed and displayed on the first picture image so thatan edge included in the narrow-band image is overlapped with an edge ofthe first picture image in display, and which has transparency” areassociated with each other. This distribution may be performed bydistributing the data structure through the internet or the like or maybe performed by selling, transferring, or lending a portable recordingmedium such as a DVD and a CD-ROM in which the data structure isrecorded.

Further, the data structure may be distributed which contains first datarepresenting an “image which contains a spatial frequency component anabsolute value of which is larger than zero and has a temporal frequencyan absolute value of which is equal to or smaller than a first value”and second data representing a “luminance motion component correspondingto the image and having a temporal frequency an absolute value of whichis equal to or larger than a second value”. This distribution may beperformed by distributing the data structure through the internet or thelike or may be performed by selling, transferring, or lending a portablerecording medium such as a DVD and a CD-ROM in which the data structureis recorded. A device received this data structure inputs the “firstdata” and the “second data” into an arithmetic unit, the arithmetic unitperforms calculation for putting the “luminance motion component” on the“image” so as to obtain a moving image, and this moving image isdisplayed from a display unit. Alternatively, the first data is inputinto an output unit, the output unit displays the “image” on a surfaceof the “object”, the second data is input into a projection unit, andthe projection unit projects the “luminance motion component” onto the“image” displayed on the surface of the “object”. Accordingly, a movingimage providing a dynamic illusion can be displayed on the display unitor the object. Further, the data structure of a video may be distributedwhich is used in a device which superimposes a video on an “object” inorder that the “object” will be perceived as if the “object” were givena motion. Here, an absolute value of a temporal frequency of theluminance motion component in several pieces of frames in a video islarger than an absolute value of a temporal frequency in several piecesof frames in a video corresponding to the “object”. Alternatively, thisdata structure may contain a luminance motion component which isobtained by subtracting luminance static components, which are obtainedfrom a static image based on a video corresponding to the “object”, fromrespective luminance components of several pieces of frames in a videocorresponding to the “object”.

INDUSTRIAL APPLICABILITY

The present invention is applicable to (1) an advertisement field inwhich an impression of a motion is added to a paper medium, animpression of a motion is given to a signboard, or the like, by lightprojection (2) a field of interior design in which design of interiorobjects such as a floor and a wall is deformed illusorily, (3) fields ofart, toys, and entertainment in which a motion is given to anillustration of a character, fusion with the conventional projectionmapping technique is performed, or the like, for example.

What is claimed is:
 1. A video generation device comprising: a firstprocessor which obtains an image, the image including a spatialfrequency component an absolute value of which is larger than zero andhaving a temporal frequency an absolute value of which is equal to orsmaller than a first value, from an input video; and a second processorwhich obtains a luminance motion component having a temporal frequencyan absolute value of which is equal to or larger than a second value,wherein the luminance motion component is a component corresponding tothe image and the second value is larger than the first value.
 2. Avideo generation device comprising: an image extractor which obtains astatic image from an input video; and a luminance motion componentextractor which subtracts a luminance static component, the luminancestatic component being obtained from the static image, from a luminancecomponent of each of several pieces of frames in the input video so asto obtain a luminance motion component.
 3. A non-transitory computerreadable recording medium having stored therein a program for making acomputer function as the video generation device according to any one ofclaim 1 or
 2. 4. A video generation device comprising: an imageextractor which obtains a static image from an input video; a luminancemotion component extractor which subtracts a luminance static componentfor frames in the input video from luminance components of the frames soas to obtain a luminance motion component, the luminance staticcomponent being obtained from the static image; and an arithmetic unitwhich generates a second video mainly including the luminance motioncomponent obtained for the frames in the input video.
 5. The deviceaccording to claim 4, wherein the arithmetic unit generates the secondvideo including only the luminance motion component obtained for theframes in the input video.
 6. A video generation method comprising: animage extracting step of obtaming a static image from an input video; aluminance motion component extracting step of subtracting a luminancestatic component for frames in the input video from luminance componentsof the frames so as to obtain a luminance motion component, theluminance static component being obtained from the static image; and anarithmetic step of generating a second video mainly including theluminance motion component obtained for the frames in the input video.7. The method according to claim 6, wherein the arithmetic stepgenerates the second video including only the luminance motion componentobtained for the frames in the input video.
 8. A non-transitory computerreadable recording medium having stored therein a program for making acomputer function as the video generation device according to any one ofclaim 4 or 5.